sync cibuildwheel test env with uv by kevinjqliu · Pull Request #2913 · apache/iceberg-python

kevinjqliu · 2026-01-14T18:03:51Z

Rationale for this change

Follow up to #2912

CI still fails with
https://github.com/apache/iceberg-python/actions/runs/21004478697/job/60383069033

Error: cibuildwheel: Command ['C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-0tbpuziz\\cp310-win_amd64\\venv-test\\Scripts\\pip.EXE', 'install', 'pytest==7.4.2', 'moto==5.0.1', 'pytest-lazy-fixture==0.6.3', 'sqlalchemy>=2.0.18,<3'] failed with code 1. 

Error: Process completed with exit code 1.

This PR fixes syncs the env with uv so we dont need to explicitly specify the deps here

Are these changes tested?

Yes
https://github.com/apache/iceberg-python/actions/runs/21007310527

Are there any user-facing changes?

kevinjqliu · 2026-01-14T18:05:09Z

I wasnt able to run #2912 since the branch was not on the apache repo. Just pushed this branch to the apache repo and ran the nightly CI with the branch https://github.com/apache/iceberg-python/actions/runs/21004675670

kevinjqliu · 2026-01-14T18:34:22Z

Awesome, it worked https://github.com/apache/iceberg-python/actions/runs/21005261083

kevinjqliu · 2026-01-14T18:34:44Z

cc @nssalian

nssalian · 2026-01-14T18:39:18Z

Thanks for the fix @kevinjqliu . Looks good.

jayceslesar · 2026-01-14T19:01:05Z

Why do uv pip install --group dev instead of uv sync --group dev ?

Wouldnt that differ vs the lockfile? I come from a mostly pdm background so unsure exactly whats happening but assuming the semantics are at least similar

kevinjqliu · 2026-01-14T19:22:28Z

good point @jayceslesar let me try that

kevinjqliu · 2026-01-14T20:01:56Z

that works! https://github.com/apache/iceberg-python/actions/runs/21007310527

.github/workflows/pypi-build-artifacts.yml

This reverts commit 2be8927.

geruh

LGTM!

Fokko

Nice, thanks @kevinjqliu!

Thanks for fixing this, which is probably not the most fun thing to do. I left one comment, apart from that it looks good to me 👍

Fokko · 2026-01-15T19:40:37Z

pyproject.toml

    "mypy-boto3-glue>=1.28.18",
    "mypy-boto3-dynamodb>=1.28.18",
    "pyarrow-stubs>=20.0.0.20251107", # Remove when pyarrow >= 23.0.0 https://github.com/apache/arrow/pull/47609
+    "sqlalchemy>=2.0.18,<3",


I guess that this is for the SqlCatalog. Wouldn't it be cleaner to see if we can remove those direct imports (probably in the conftest.py)?

yep it is for the SqlCatalog. Now that we have SqlCatalog in conftest.py, this is technically a new "dev dependency"

iceberg-python/tests/conftest.py

Lines 2965 to 2972 in 8e4d424

def _create_sql_catalog(name: str, warehouse: Path) -> SqlCatalog:

catalog = SqlCatalog(

name,

uri="sqlite:///:memory:",

warehouse=f"file://{warehouse}",

)

catalog.create_tables()

return catalog

Yes, but it isn't being used for this specific test:

CIBW_TEST_COMMAND: "uv run --directory {project} pytest tests/avro/test_decoder.py"

Once the Cython parser is out, we can rip all this out :D

kevinjqliu · 2026-01-16T00:06:54Z

Thanks for the review, everyone!

fix quoting

c7d6ac3

kevinjqliu added 3 commits January 14, 2026 10:09

do we need sqlalchemy?

e9fd5c4

lets try uv

01f5690

include sqlalchemy in dev dep

070e536

kevinjqliu changed the title ~~fix quoting~~ sync cibuildwheel test env with uv Jan 14, 2026

kevinjqliu mentioned this pull request Jan 14, 2026

nightly build failing #2911

Closed

3 tasks

kevinjqliu temporarily deployed to testpypi January 14, 2026 18:33 — with GitHub Actions Inactive

kevinjqliu requested a review from Fokko January 14, 2026 18:34

nssalian approved these changes Jan 14, 2026

View reviewed changes

kevinjqliu added 2 commits January 14, 2026 11:22

update uv lock

2d44374

try uv sync

42f72a0

kevinjqliu added 2 commits January 14, 2026 11:29

uv run pytest

b33abd6

relative path

f1409e1

kevinjqliu temporarily deployed to testpypi January 14, 2026 19:42 — with GitHub Actions Inactive

jayceslesar approved these changes Jan 14, 2026

View reviewed changes

geruh reviewed Jan 14, 2026

View reviewed changes

.github/workflows/pypi-build-artifacts.yml Show resolved Hide resolved

kevinjqliu added 2 commits January 14, 2026 13:58

--all-extras

2be8927

Revert "--all-extras"

be10d20

This reverts commit 2be8927.

geruh approved these changes Jan 14, 2026

View reviewed changes

kevinjqliu temporarily deployed to testpypi January 15, 2026 02:20 — with GitHub Actions Inactive

Fokko approved these changes Jan 15, 2026

View reviewed changes

kevinjqliu merged commit 832f83d into apache:main Jan 16, 2026
16 checks passed

kevinjqliu deleted the kevinjqliu/fix-nightly branch January 16, 2026 00:06

	def _create_sql_catalog(name: str, warehouse: Path) -> SqlCatalog:
	catalog = SqlCatalog(
	name,
	uri="sqlite:///:memory:",
	warehouse=f"file://{warehouse}",
	)
	catalog.create_tables()
	return catalog

Conversation

kevinjqliu commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

Uh oh!

kevinjqliu commented Jan 14, 2026

Uh oh!

kevinjqliu commented Jan 14, 2026

Uh oh!

kevinjqliu commented Jan 14, 2026

Uh oh!

nssalian commented Jan 14, 2026

Uh oh!

jayceslesar commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevinjqliu commented Jan 14, 2026

Uh oh!

kevinjqliu commented Jan 14, 2026

Uh oh!

Uh oh!

geruh left a comment

Choose a reason for hiding this comment

Uh oh!

Fokko left a comment

Choose a reason for hiding this comment

Uh oh!

Fokko Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

kevinjqliu Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fokko Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Fokko Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kevinjqliu commented Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kevinjqliu commented Jan 14, 2026 •

edited

Loading

jayceslesar commented Jan 14, 2026 •

edited

Loading

kevinjqliu Jan 16, 2026 •

edited

Loading